Brain decoder

A study published today in journal Nature introduced a non-invasive semantic decoder based on functional magnetic resonance imaging (fMRI) and technology similar to the one powering ChatGPT.

The decoder can convert a person's brain activity when they are listening to or silently imagine telling a story into a continuous stream of text. The method created by the team from the University of Texas at Austin may enable people to speak with their minds. 

Given novel brain recordings, this decoder generates intelligible word sequences that recover the meaning of perceived speech, imagined speech and even silent videos, demonstrating that a single decoder can be applied to a range of tasks. And it can fruitfully work with multiple brain regions. This demonstrates the viability of non-invasive language brain–computer interfaces. 

Computational task was performed by Generative Pre-trained Transformer (GPT), a 12-layer neural network using multi-head self-attention to combine representations of each word in a sequence with representations of previous words. GPT was trained on a large corpus of books to predict the probability distribution over the next word n a sequence. The authors fine-tuned GPT on a corpus comprising Reddit comments (over 200 million total words) and 240 autobiographical stories from The Moth Radio Hour and Modern Love. The model was trained for 50 epochs with a maximum context length of 100. 


REFERENCES

Tang, J., LeBel, A., Jain, S. et al. Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci (2023). https://doi.org/10.1038/s41593-023-01304-9

Jerry Tang, Amanda LeBel, Shailee Jain, Alexander G. Huth. Semantic reconstruction of continuous language from non-invasive brain recordings bioRxiv 2022.09.29.509744; doi: https://doi.org/10.1101/2022.09.29.509744. download pdf

Comments

Post a Comment

Popular posts from this blog

Precision Nutrigeroscience: Nutrition for Longevity and Brain Health

Image
The pursuit of extended lifespan and improved health requires a deep understanding of the complex interplay between our diet and unique biological factors.  This is where precision nutrigeroscience emerges as a transformative approach, leveraging the power of biomarker identification and analysis to tailor dietary interventions. It embraces the diversity of human physiology, acknowledging that what works for one may not work for another, especially when it comes to nutrition. The varying impact of carbohydrates and fats on mortality is a testament to this complexity. Meat intake is linked to higher life expectancies, but the impact of carbohydrates and fats varies among individuals. While some thrive on higher carbohydrate diets, others may benefit from reduced carb intake. Similarly, the type of fats consumed (animal vs. plant-based) plays a significant role in health outcomes. The goal of precision nutrigeroscience is to decipher these individual needs and craft dietary recommendatio
Read more

HippoCamera for Aging Brain

Image
Memory decline is common in aging and significantly worsens quality of life, but there are few, if any, effective interventions to improve episodic memory recall in older adults. And there are no effective interventions specifically targeting autobiographical episodic memory. A smartphone app called HippoCamera allows users to create memory cues of specific events that they personally value and hope to remember. These cues consist of a short verbal description and a short video recording of the event. The approach was shown to be effective tool for improving the recall of episodic memories in older adults. Seniors were able to improve their ability to recall episodic memories, increase the differentiation of hippocampal activity, and better remember the personal past. These findings suggest that using mobile phone technologies to record and replay episodic memories may be a feasible, low- cost, and effective way to improve memory recall in older adults (Adapted from SpinnerChef:  Spinn
Read more

Brain vs RTX and LLMs: A Computational Showdown

Image
In the age of rapidly advancing technology, comparing the human brain to building blocks of a computer opens up an intriguing conversation. Both systems represent incredible feats of engineering—one through billions of years of evolution, the other through decades of technological innovation.  This image  that I came across on reddit  illustrates features of the brain vs a powerful graphics card like the NVIDIA RTX 4090. As of September 2024, the NVIDIA GeForce RTX 4090 remains one of the most powerful consumer GPUs available, especially for gaming and productivity tasks like rendering and AI workloads. It excels in 4K gaming and can handle even the most graphically demanding games, with exceptional performance in ray tracing and DLSS 3 technology, delivering up to 3-4x the performance of previous-generation GPUs like the RTX 3090 Ti. NVIDIA GeForce RTX 4090 is rumored to be  discontinued in October in preparation For Next-Gen RTX 5090 & 5090D GPUs, which could be up to 70% faster
Read more